CLAIMS 

[1] An endoscope system for taking images of an inside of an 
object, comprising: 

a camera operable to take images of the inside of the object 
5 in a living body, which is capable of motion; and 

an image generation unit operable to generate a panoramic 
image of the inside of the object by performing a video mosaicking 
process, a motion correction process, and an image modification 
process through energy minimization on the plurality of images 
10 obtained by said camera, said processes being intended for pasting 
the images, estimating camera motion, correcting previously 
definable motion in the living body and correcting previously 
indefinable internal deformation in the living body. 

15 [2] The endoscope system according to claim 1, 
wherein said image generation unit includes: 
a motion estimation unit operable to estimate the motion of 

said camera based on the images of the inside of the object taken by 

said camera; and 

20 a panoramic image generation unit operable to generate the 

panoramic image of the inside of the object from the images taken 
by said camera based on the estimation result for the motion of said 
camera. 

25 [3] The endoscope system according to claim 2, 
wherein said motion estimation unit includes: 
a corresponding point detection unit operable to express an 
observation point of said camera in a spherical coordinate system 
and detect corresponding points for the observation point from the 
30 plurality of images obtained by said camera; and 

a motion parameter estimation unit operable to estimate a 
motion parameter expressing the motion of said camera based on a 
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plurality of the corresponding points. 

[4] The endoscope system according to claim 2, 
wherein said motion estimation unit includes: 
a camera motion estimation unit operable to estimate the 
motion of said camera from two temporally different images 
obtained by said camera by using an epipolar constraint condition; 
and 

a camera motion correction unit operable to correct the 
motion of said camera estimated by said camera motion estimation 
unit by performing a bundle adjustment process using the plurality 
of temporally different images obtained by said camera. 

[5] The endoscope system according to claim 4, 
15 wherein said camera motion correction unit is operable to 

perform the bundle adjustment process by approximating a change 
inside the object due to segmentation movement by a sine wave. 

[6] The endoscope system according to claim 4, 
20 wherein said camera motion correction unit is operable to 

perform the bundle adjustment process by approximating a change 
inside the object due to peristalsis movement by movement of a 
soliton. 

25 [7] The endoscope system according to claim 4, 
wherein the object is a cylindrical object, and 
said panoramic image generation unit is operable to 
generate a cylindrical-shaped model of the object and fit the 
plurality of images obtained by said camera to the 
30 cylindrical-shaped model based on a feature point used at the time 
of estimating the motion. 
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[8] The endoscope system according to claim 2, further 
comprising 

a position/attitude sensor operable to measure a 
self-position or attitude, 
5 wherein said motion estimation unit is operable to estimate 

the motion of said camera in consideration of a measurement result 
by said position/attitude sensor. 

[9] The endoscope system according to claim 1, 
10 wherein said image generation unit includes: 

a feature region cutout unit operable to cut out a plurality of 

feature regions having a predetermined size from each of the 

plurality of images obtained by said camera; and 

a panoramic image generation unit operable to define 
15 predetermined energy based on the plurality of feature regions 

included in each of the plurality of images, associate the plurality of 

feature regions between the plurality of images such that the energy 

is minimized, and generate a panoramic image of the inside of the 

object based on the association result. 

20 

[10] The endoscope system according to claim 9, 

wherein the predetermined energy is determined based on 
differences in pixel value between the plurality of feature regions 
included in each of two temporally successive images. 

25 

[11] The endoscope system according to claim 9, 

wherein the predetermined energy is determined based on 
differences in area between triangular patches obtained by 
connecting the plurality of feature regions included in each of two 
30 temporally successive images. 

[12] The endoscope system according to claim 9, 
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wherein said camera is an omnidirectional camera, and 
the predetermined energy is determined based on a 
difference between (i) a coordinate obtained by correcting, based on 
a movement component of said omnidirectional camera, a 
5 coordinate of a great circle which appears in an image after a first 
image taken by said omnidirectional camera is transformed in a 
spherical coordinate system with its center at a viewpoint of said 
omnidirectional camera, and (ii) a coordinate of a great circle which 
appears in an image after a second image temporally successive to 
10 the first image and taken by said omnidirectional camera is 
transformed in the spherical coordinate system. 

[13] The endoscope system according to claim 9, 

wherein the predetermined energy is determined based on a 
is degree of deviation of a plurality of control points, in a second image 
taken by said camera, which respectively correspond to a plurality of 
control points selected from a first image taken by said camera. 

[14] The endoscope system according to claim 9, 
20 wherein the predetermined energy is determined based on a 

degree of deviation between a plurality of control points selected 
from a first image taken by said camera and a plurality of control 
points, in a second image taken by said camera, which respectively 
correspond to the plurality of control points selected from the first 
25 image. 

[15] The endoscope system according to claim 9, 

wherein the plurality of feature regions are regions, among 
the plurality of regions having a predetermined size included in each 
30 of the images, in which the squared sum of derivatives of pixel 
values is greater than a predetermined threshold value. 
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[16] The endoscope system according to claim 1, 

wherein said camera is an omnidirectional camera, and 
said image generation unit is operable to generate a 

panoramic image having a fixed visual angle with respect to a 
5 direction perpendicular to a traveling direction of said 

omnidirectional camera by performing a video mosaicking process 

on the plurality of images obtained by said omnidirectional camera. 

[17] The endoscope system according to claim 1, 
10 wherein said camera is mounted on a tip of a probe that is to 

be inserted into a digestive organ. 

[18] The endoscope system according to claim 1, 

wherein said camera is enclosed in a capsule that can be 
15 swallowed by a human or an animal. 
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